Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

从消息传递机制中受益，图形神经网络（GNN）在图形数据上的繁荣任务上已经成功。但是，最近的研究表明，攻击者可以通过恶意修改图形结构来灾难性地降低GNN的性能。解决此问题的直接解决方案是通过在两个末端节点的成对表示之间学习度量函数来建模边缘权重，该指标函数试图将低权重分配给对抗边缘。现有方法使用监督GNN学到的原始功能或表示形式来对边缘重量进行建模。但是，两种策略都面临着一些直接问题：原始特征不能代表节点的各种特性（例如结构信息），而受监督的GNN学到的表示可能会遭受分类器在中毒图上的差异性能。我们需要携带特征信息和尽可能糊状的结构信息并且对结构扰动不敏感的表示形式。为此，我们提出了一条名为stable的无监督管道，以优化图形结构。最后，我们将精心设计的图输入到下游分类器中。对于这一部分，我们设计了一个高级GCN，可显着增强香草GCN的鲁棒性，而不会增加时间复杂性。在四个现实世界图基准上进行的广泛实验表明，稳定的表现优于最先进的方法，并成功防御各种攻击。

近年来，在移动设备上部署深度学习（DL）一直是一个显着的趋势。为了支持对开发DL的快速推断，DL库作为算法和硬件扮演着至关重要的角色。不幸的是，先前的工作从未深入现代DL Libs的生态系统，并为其性能提供定量结果。在本文中，我们首先建立了一个全面的基准，其中包括6个代表性DL LIB和15种多元化的DL模型。然后，我们在10个移动设备上进行了广泛的实验，这有助于揭示当前移动DL LIBS生态系统的完整景观。例如，我们发现表现最佳的DL LIB在不同的模型和硬件中严重碎片，这些DL Libs之间的差距可能相当巨大。实际上，DL LIB的影响会淹没算法或硬件的优化，例如模型量化和基于GPU/DSP的异质计算。最后，在观察结果上，我们总结了对DL Lib生态系统中不同角色的实际意义。

交叉语言信息检索（CLIR）旨在将以与用户查询不同的语言编写的文档进行排序。不同语言之间的内在差距是CLIR的基本挑战。在本文中，由于多种语言的实体的足够信息，我们将多语言知识图（kg）引入CLIR任务。它被视为“银弹”，同时在查询和文档之间进行显式对齐，并扩大查询的表示。我们提出了一个名为CLIR的模型，为我们的任务提供了分层知识增强（加息）。所提出的模型用多语言BERT编码查询，文档和kg中的文本信息，并在具有分层信息融合机制中将kg信息包含在查询文件匹配过程中。特别是，徒步旅行首先将kg中的实体及其社区集成到具有知识级融合的查询表示中，然后将来自源语言的知识结合起来进一步减轻语言级融合的语言间隙。最后，实验结果表明，徒步旅行达到了最先进的竞争对手的大量改进。

具有释义生成的长期问题是如何获得可靠的监督信号。在本文中，我们基于假设产生与鉴定相同的上下文相同的含义的两个句子的概率应该是相同的，提出了一种无监督的范例。灵感来自这一基本因的主意，我们提出了一种流水线系统，该系统由基于上下文语言模型的候选候选生成组成，使用评分函数的候选滤波，以及基于所选候选者的释放模型训练。提议的范例提供了现有的释义生成方法的优点：（1）使用上下文规范器在含义上，该模型能够产生大量的高质量释义对; （2）使用人为可解释的评分功能来选择来自候选者的释义对，所提出的框架为开发人员提供了一种与数据生成过程进行干预的通道，导致更可控的模型。不同任务和数据集的实验结果表明，拟议模型在监督和无人监督的设置中的有效性。

The frustratingly fragile nature of neural network models make current natural language generation (NLG) systems prone to backdoor attacks and generate malicious sequences that could be sexist or offensive. Unfortunately, little effort has been invested to how backdoor attacks can affect current NLG models and how to defend against these attacks. In this work, by giving a formal definition of backdoor attack and defense, we investigate this problem on two important NLG tasks, machine translation and dialog generation. Tailored to the inherent nature of NLG models (e.g., producing a sequence of coherent words given contexts), we design defending strategies against attacks. We find that testing the backward probability of generating sources given targets yields effective defense performance against all different types of attacks, and is able to handle the {\it one-to-many} issue in many NLG tasks such as dialog generation. We hope that this work can raise the awareness of backdoor risks concealed in deep NLG systems and inspire more future work (both attack and defense) towards this direction.

In contrast to the control-theoretic methods, the lack of stability guarantee remains a significant problem for model-free reinforcement learning (RL) methods. Jointly learning a policy and a Lyapunov function has recently become a promising approach to ensuring the whole system with a stability guarantee. However, the classical Lyapunov constraints researchers introduced cannot stabilize the system during the sampling-based optimization. Therefore, we propose the Adaptive Stability Certification (ASC), making the system reach sampling-based stability. Because the ASC condition can search for the optimal policy heuristically, we design the Adaptive Lyapunov-based Actor-Critic (ALAC) algorithm based on the ASC condition. Meanwhile, our algorithm avoids the optimization problem that a variety of constraints are coupled into the objective in current approaches. When evaluated on ten robotic tasks, our method achieves lower accumulated cost and fewer stability constraint violations than previous studies.

A storyboard is a roadmap for video creation which consists of shot-by-shot images to visualize key plots in a text synopsis. Creating video storyboards however remains challenging which not only requires association between high-level texts and images, but also demands for long-term reasoning to make transitions smooth across shots. In this paper, we propose a new task called Text synopsis to Video Storyboard (TeViS) which aims to retrieve an ordered sequence of images to visualize the text synopsis. We construct a MovieNet-TeViS benchmark based on the public MovieNet dataset. It contains 10K text synopses each paired with keyframes that are manually selected from corresponding movies by considering both relevance and cinematic coherence. We also present an encoder-decoder baseline for the task. The model uses a pretrained vision-and-language model to improve high-level text-image matching. To improve coherence in long-term shots, we further propose to pre-train the decoder on large-scale movie frames without text. Experimental results demonstrate that our proposed model significantly outperforms other models to create text-relevant and coherent storyboards. Nevertheless, there is still a large gap compared to human performance suggesting room for promising future work.

Retrieval-augmented in-context learning has emerged as a powerful approach for addressing knowledge-intensive tasks using frozen language models (LM) and retrieval models (RM). Existing work has combined these in simple "retrieve-then-read" pipelines in which the RM retrieves passages that are inserted into the LM prompt. To begin to fully realize the potential of frozen LMs and RMs, we propose Demonstrate-Search-Predict (DSP), a framework that relies on passing natural language texts in sophisticated pipelines between an LM and an RM. DSP can express high-level programs that bootstrap pipeline-aware demonstrations, search for relevant passages, and generate grounded predictions, systematically breaking down problems into small transformations that the LM and RM can handle more reliably. We have written novel DSP programs for answering questions in open-domain, multi-hop, and conversational settings, establishing in early evaluations new state-of-the-art in-context learning results and delivering 37-200%, 8-40%, and 80-290% relative gains against vanilla LMs, a standard retrieve-then-read pipeline, and a contemporaneous self-ask pipeline, respectively.

Future work sentences (FWS) are the particular sentences in academic papers that contain the author's description of their proposed follow-up research direction. This paper presents methods to automatically extract FWS from academic papers and classify them according to the different future directions embodied in the paper's content. FWS recognition methods will enable subsequent researchers to locate future work sentences more accurately and quickly and reduce the time and cost of acquiring the corpus. The current work on automatic identification of future work sentences is relatively small, and the existing research cannot accurately identify FWS from academic papers, and thus cannot conduct data mining on a large scale. Furthermore, there are many aspects to the content of future work, and the subdivision of the content is conducive to the analysis of specific development directions. In this paper, Nature Language Processing (NLP) is used as a case study, and FWS are extracted from academic papers and classified into different types. We manually build an annotated corpus with six different types of FWS. Then, automatic recognition and classification of FWS are implemented using machine learning models, and the performance of these models is compared based on the evaluation metrics. The results show that the Bernoulli Bayesian model has the best performance in the automatic recognition task, with the Macro F1 reaching 90.73%, and the SCIBERT model has the best performance in the automatic classification task, with the weighted average F1 reaching 72.63%. Finally, we extract keywords from FWS and gain a deep understanding of the key content described in FWS, and we also demonstrate that content determination in FWS will be reflected in the subsequent research work by measuring the similarity between future work sentences and the abstracts.